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Abstract: 


Hiring presents an important opportunity for districts to find educators with values and 
beliefs that align with district goals. Yet beliefs about how to best address persistent challenges 
like inequality are difficult to measure. We use administrative data from over 10,000 applications 
to certificated positions in an urban California school district and structural topic modeling 
(STM) to code applicant essays about closing achievement gaps, identify their beliefs, and test 
how they influence application behavior and hiring. Compared to white applicants, Hispanic and 
African American applicants are more likely to identify structural causes for inequities and 
discuss educators’ responsibilities for addressing inequality by challenging biases, as well as 
experiences with cultural and linguistic diversity. Similar themes emerge among applicants 
applying to and being hired in schools with more marginalized students. Techniques like STM 
can decipher hard-to-measure beliefs from administrative data, providing valuable information 
for hiring and decision-making. 
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Schools across the country serve an increasingly diverse student body, and California’s 
schools lead the nation in diversity. Currently three-quarters of public school students in 
California identify as a race/ethnicity other than white and nearly 43 percent speak a language 
other than English at home (California Department of Education 2016a, 2016b). Yet these 
students have not had equal educational opportunities or successes. In spite of repeated efforts to 
equalize educational experiences and outcomes for students, school segregation and racial 
achievement gaps persist and economic achievement gaps have widened (Harris and Herrington 
2006; Reardon 2011; Reardon and Bischoff 2011; Reardon and Owens 2014). 

Hiring presents school districts with an opportunity to identify educators who not only 
have relevant experience, but also possess the beliefs, attitudes, and pedagogical skills that 
enable them to promote engagement and achievement among students with varied experiences 
and backgrounds. However, districts often struggle to identify candidates with the characteristics 
that they most desire, and identifying candidates whose beliefs align best with district policy 
strategies and goals is especially challenging. Although recent evidence suggests that well- 
developed applications and interview processes can provide valuable insights into applicant 
quality, such hiring methods can be cost prohibitive for many districts (Jacob et al. 2016; 
Rockoff et al. 2011). 

This study examines how educator applicants address issues of equity and diversity in 
their application materials and considers how such beliefs and attitudes influence application 
decisions and hiring outcomes. We use seven years of unique essay data from over ten thousand 
applications to our partner district in California to examine how educator applicants express their 
beliefs and how the district evaluates them in its hiring process.’ Because equity is central to the 


1 We include applicants to all certificated positions, including classroom teachers, principals, therapists, and others, 
because some applicants apply to both teaching positions as well as some other type of certificated position for 


district’s mission, it explicitly asks all applicants to write a short essay discussing how they 
would address the achievement gap in their classrooms if hired. We use Structural Topic 
Modeling (STM) to code the ten most common essay themes from applicant responses to this 
prompt. We describe variation in the prevalence of each essay theme across educators and 
contrast response themes by educator race. We then use regression analyses to test whether 
applicants that articulate the most common beliefs, values, and strategies about how to address 
achievement inequities apply to schools with more disadvantaged students, rate more highly in 
application review, and are ultimately more likely to be hired by the district. 

Educator short-essay responses cover a wide range of topics, some expressing more 
general sentiments about the achievement gap and others describing specific strategies and 
beliefs about how they would address it. Essay content varies by race/ethnicity, with African 
American and Hispanic applicants focusing more on naming structural causes of inequities 
facing students and advocating solutions rooted in educators’ responsibilities to address inequity 
and their own biases and prioritize cultural and linguistic diversity, and Asian applicants 
focusing more on building supportive classrooms and family and community engagement, 
relative to white applicants. Several of the essay themes differentially predict applicants’ scores 
on rubric evaluations of their applications and the likelihood that the district hires them. 
Compared to a common reference essay theme—differentiated instruction—applicant essays that 
describe experience with cultural and linguistic diversity and educators’ responsibilities to 


address inequities receive the highest rubric scores and are associated with the highest likelihood 


which they believe they are qualified, making the search processes for both positions contingent upon one another. 
For parsimony, throughout the manuscript we refer to all applicants as “educators” or “teachers” reflecting the fact 
that the majority of the existing literature on educator hiring focuses on teachers. Results discussed below are largely 
consistent whether we use the full sample of educator applicants or only the subsample of applicants to teaching 
positions (81 percent of the full applicant pool and 85 percent of the hired pool). Nearly all coefficients are similar in 
sign, significance, and magnitude. A small subset lose or gain significance with the teacher only sample. 


of being hired in the district. Furthermore, several of these themes are positively related to 
applying to and being hired in schools with the largest populations of traditionally underserved 
students—particularly Hispanic students and English Language Learners—compared with 
applicants who write about other topics. These results are robust to a variety of model 
specifications including models that control for the writing quality of the essays, essay sentiment, 
educator demographics, experience, and credentials, and fixed effects for the specific jobs to 
which applicants applied. 

Structural Topic Modeling (STM) techniques present a promising tool for examining 
administrative data from school districts as well as in other settings, particularly qualitative and 
long-form data that is typically too resource-intensive to analyze. In this specific application and 
elsewhere, STM can help to examine hard to measure practices and beliefs, providing more 
information at the point of hire and informing decision-making in other areas. 

Background and Motivation 

Though somewhat diminished in recent decades, racial achievement gaps remain an 
enduring feature of schools nationwide (Gamoran 2001; Murphey 2014; Reardon 2011). 
Economic achievement gaps, meanwhile, are expanding markedly, especially between those in 
the middle of the distribution and their highest income peers (Reardon 2011). School segregation 
remains entrenched and perpetuates unequal learning opportunities along racial/ethnic and 
economic lines (Reardon and Owens 2014; Reardon and Bischoff 2011). Our society expects 
educators to address these persistent challenges and to create classrooms that provide 
opportunities for all students. However, because we struggle to agree on the goals of schooling, 


we have no singular blueprint for how to do so (Labaree 1997). 


Instead, teachers have a variety of goals for their teaching, and some may not prioritize 
reducing inequality (Rimm-Kaufman et al. 2006). If educators in some schools think addressing 
inequality is a key priority of their work while educators in other schools do not, they might 
create very different learning environments for their students, further exacerbating inequality. 
Educators also have differing levels of comfort and experience in planning and enacting 
solutions to address inequality in their work. Understanding this variation is important because 
teachers’ beliefs are an important indicator of what they are likely to do in their classrooms 
(Buehl and Beck 2015; Pajares 1992; Wilkins 2008; Opfer and Pedder 2011). In spite of the 
central role we expect educators to play in addressing inequality, we know little about what 
educators think about inequality and how they should work to address it. 

We need a better understanding of what educators think about inequality because 
educators’ identities and beliefs are consequential for learning environments, opportunities, and 
student outcomes, particularly for marginalized students. We know that teachers’ identities shape 
students’ schooling experiences and outcomes. For example, teacher-student race and gender 
congruence positively impacts student achievement, grades, their academic perceptions and 
attitudes, and discipline outcomes (Dee 2004; Egalite and Kisida 2018; Egalite, Kisida, and 
Winters 2015; Fox 2016; Gershenson et al. 2017; Lindsay and Hart 2017). These effects are 
often particularly large for black students. The mechanisms driving these effects—typically 
presumed rather than empirically tested—are that same-race or same-gender educators not only 
offer representation but convey values and attitudes that recognize and affirm the unequal 
experiences of students from marginalized backgrounds. 

Underneath demographic similarities, educators have a wide variety of experiences and 


knowledge that ultimately shape their beliefs about their work and their students. Teacher beliefs 


are an integral part of teacher practice, learning, development, and identity (Hollingsworth 1989; 
Opfer and Pedder 2011; Smagorinsky et al. 2004). Teachers’ perceptions affect how they treat 
their students, the types of help and support they provide, and teachers’ empathy towards 
students (Calarco 2011; Ferguson 2003; Okonofua, Paunesku, and Walton 2016). Teacher 
perceptions, expectations, and stereotypes can influence disciplinary practices and students’ later 
course trajectories and achievement, contributing to the growth of achievement gaps (Ferguson 
2003; Okonofua and Eberhardt 2015; Baker et al. 2015; Alvidrez and Weinstein 1999). 
Teachers’ beliefs about students’ social disadvantage influence the relationship between 
teachers’ attitudes and student achievement (Rochmes 2018). Moreover, teachers’ expectations 
for student ability and achievement matter most for the outcomes of students from disadvantaged 
groups (Dee 2005; Downey and Pribesh 2004; Ferguson 2003; McGrady and Reynolds 2013; 
Rist 1970). 

Understanding what educators think about how to address inequality is important not 
only because of how beliefs influence educator practice, but also because if educators’ goals are 
misaligned with school or district goals, policy solutions to reduce inequality are unlikely to 
work. Teachers can promote district goals, but can also impede and even derail them if they are 
not well-aligned with the goals of individual teachers or the collective goals of their instructional 
teams (Coburn 2001, 2004; Coburn, Hill, and Spillane 2016; Golann 2017; Spillane 1999). 

Because educators play such an important role in carrying out district priorities, the 
alignment between district goals and educators’ goals is an important consideration for educator 
hiring. Once we know what educators think about how to address inequality, we can also 
examine how well aligned these beliefs are with district priorities. Districts might use such data 


in a range of human resource decisions, including hiring. For districts that are particularly 


concerned with reducing opportunity and achievement gaps for diverse students, hiring presents 
an important opportunity to identify educators equipped to meet the needs of marginalized 
students. 

In response to national priorities and district interests, many districts prioritize equity- 
focused beliefs in their recruitment, demonstrating this commitment by prioritizing the 
diversification of their teaching force (Bireda and Chait 2011; Brown and Boser 2017; Villegas 
and Irvine 2010). In spite of concerted efforts to increase educator diversity through recruitment, 
training, and retention efforts, the pace of diversification is slow; in 2016, only eighteen percent 
of the nation’s teachers identified as teachers of color, an increase of only one percent from 2004 
(King, Jr., McIntosh, and Bell-Ellwanger 2016; National Center for Education Statistics 2004). 

In addition to redoubling efforts to increase educator diversity, there may be other 
avenues for making learning opportunities and outcomes more equitable by carefully considering 
the beliefs and values of all prospective educators, regardless of demographic background. 
Districts interested in prioritizing candidates with particular beliefs and values can use interview 
and application essays to collect this information. Earlier research indicated that applications and 
interviews could not help districts make effective judgments to fulfill hiring priorities, but recent 
evidence suggests that thoughtful, well-targeted recruitment efforts can provide meaningful 
information about applicants (Balter and Duncombe 2006; DeArmond, Gross, and Goldhaber 
2010; Jacob et al. 2016; Rockoff et al. 2011). In particular, careful consideration of applicant 
characteristics through rigorous screening methods and interview processes can identify teachers 
with desired characteristics and behaviors that positively impact student achievement 


(Goldhaber, Grout, and Huntington-Klein 2016; Jacob et al. 2016). 


If districts want to develop a teaching force that has competencies, beliefs, and mindsets 
that align with district goals for addressing inequality, then they need to create a recruitment 
process that can identify these qualities among its applicants. This study draws on data from a 
partner district in California that uses its application to examine educators’ approaches to equity 
goals during its recruiting process. In particular, it asks all applicants to respond to a short- 
answer essay prompt asking about the approaches they will use to address achievement gaps in 
the district. The district’s human resources department scores these essays using a rubric during 
an initial screening process, reviewing for general evidence and understanding of equity 
commitments rather than specific beliefs, attitudes, or teaching strategies. This limited 
consideration is understandable given that application review is a time-intensive process, 
particularly with a large volume of applications. 

Given resource constraints, it is difficult to measure educators’ beliefs for a large number 
of individuals in other ways. It is time consuming and expensive to conduct multi-stage 
interviews or surveys. Such methods may also provide only a narrow slice of what educators 
think and can be prone to response bias or adverse reactions to particular types of interviewers. 
For districts wanting to learn more about applicants’ beliefs, such methods can be time and cost 
prohibitive. 

Recent advances in text mining paired with improvements in district administrative data 
collected from application materials provide new avenues for learning about applicant beliefs 
outside of extensive interviews. Social science research is just beginning to make use of text and 
dialogue to reveal additional features about individuals, their conversations, and verbal and 
written statements (Bettinger, Liu, and Loeb 2016; Gentzkow, Kelly, and Taddy 2017; Kelly et 


al. 2018; Liu 2018; McFarland, Jurafsky, and Rawlings 2013). Economist Jens Ludwig (2018) 
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highlights the utility of data-mining techniques to help solve complex policy problems. 
Sociologists Roberto Fernandez and Brian Rubineau (this issue) identify important insights that 
applicant pool and referral data can provide about hiring dynamics. 

This study provides an important first case of applying machine-learning techniques to 
text-rich application data to examine the role of applicant beliefs in hiring processes and 
outcomes. It examines these relationships for more than ten thousand applicants over seven years 
in a large, urban district that has often struggled to attract and retain educators, particularly in its 
most disadvantaged schools. In addition to these methodological and data advances, this study 
has the novel purpose of focusing on educator attitudes towards and solutions for addressing the 
achievement gap, which is difficult to measure in a large sample. This study tests whether a 
district that expresses a deep commitment to equity and justice actually selects applicants that 
articulate attitudes and solutions that cohere with the district’s priorities. Together, these results 
shed light on two often-neglected pieces of the inequality puzzle—what educators think and 
whether districts act in line with their values. 

Our investigation of the role that educator beliefs about inequality play in hiring 


examines the following research questions: 


1; What themes predominate in applicants’ attitudes about achievement gaps? 

2: Do the themes discussed vary by applicant race? 

Sh Are individuals who write about particular themes more likely to apply to schools with 
particular characteristics? 

4. Do the essay themes and scores predict hiring outcomes, overall and at particular 
schools? 


Data & Methods 
This paper uses school-district administrative data to describe educator attitudes and 
examine how these attitudes affect hiring outcomes. Our source for these attitudes is short- 


answer responses on job applications. These data are conventional, in the sense that many 


11 


districts screen applicants with some type of writing exercise (Jones 2012), but our use of them is 
novel because we code them for applicants’ attitudes and dispositions using structural topic 
modeling. These data provide the first use of machine-learning coded applicant essays to learn 
more about the role of applicants’ beliefs in hiring. This district’s screening question is 
particularly useful for examining educators’ attitudes because it asks applicants to wrestle with a 
problem that will directly affect their daily teaching experiences. As a result, it provides a rich 
lens to uncover educators’ feelings about this challenge. 

Our partner district is an urban, public school district in California. It employs over 3,500 
educators and administrators to serve a diverse student body of over fifty thousand students 
(California Department of Education 2015). It is also a district with highly unequal outcomes for 
its students. In recent years, among California’s large urban districts, it had both the highest 
average achievement and the widest gap between that average and the district’s lowest- 
performing students, as well as large racial/ethnic achievement gaps (District website, 2008). 
Although the district has made progress, gaps in suspensions, graduation rates, and achievement 
are an ongoing challenge (Barba 2017; Koskey 2011; District website, 2017). 

To examine these relationships, we use applicant data from our partner district’s Human 
Resources Department. Applicants to positions in the district applied through a proprietary 
online interface. Data from this interface were then linked to other administrative records from 
human resources by the research team. Current school district and state databases increasingly 
include detailed information about employees, but it is much less common that data from 
applications to district positions are linked to other district data sources (but see Goldhaber, 
Grout, and Huntington-Klein 2016; Jacob et al. 2016; and Saavedra et al. 2017 for notable 


exceptions). This type of connection allows for new research approaches that link the applicant 
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pool to the pool of employed educators and other employees within a district. This approach has 
provided novel insights about hiring pathways, sorting, and discrimination in other sectors 
(Fernandez and Friedrich 2011; Petersen and Togstad 2006). However, existing research has not 
utilized applicant data that includes long-form writing and would typically be considered 
qualitative in nature. By applying structural topic modeling to applicant essays, this study is the 
first to link attitudes within the applicant pool to hiring outcomes. 

We examine data for all applicants to certificated positions from March of 2009 to 
October of 2015. Figure 1 shows the number of job postings, applications (in thousands), and 
applicants hired by year. 


[Figure 1 Here] 


After an initial acceleration as the system came online, postings, applications, and 
number of applicants hired stayed fairly consistent across the seven years of our panel, with an 
uptick in applications around the end of the financial crisis in 2011. During this time window, 
this district posted 11,599 unique jobs, of which we use the 6,706 positions that received one or 
more complete application. Postings received 87 applications on average (range 1-544). The 
district received 218,196 applications for all certificated positions from 14,421 unique 
individuals. Our analytic sample consists of the 10,188 applicants (13,016 applicant-year 
observations) that completed essays (164,367 applications). Applicants submitted an average of 
16 applications each over this period (an average of 22.97 at the applicant-year level). The 
district ultimately hired 2,883 individuals from this pool. Table 1 presents descriptive 


characteristics, experience, and qualifications for the district applicant pool to all certificated 


? Some applicants (27 percent) applied in multiple years, and some applicants applied and were hired in multiple 
years. These applicants have distinct application data from each job search and are thus treated separately in all 
analyses. Our sample of 2,883 hired individuals includes 2,442 unique applicants, some hired multiple times. 
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positions between 2009 and 2015, and compares these characteristics between successful and 
unsuccessful applicants. 
[Table 1 Here] 

Hired applicants differ on many dimensions from those who were not hired. More of the 
hired applicants were Asian or Latino and fewer were white.* The hired applicants had fewer 
years of experience, but were more likely to have worked in the district before applying for the 
job for which we observe them being hired. A greater proportion of hired teachers had single 
subject credentials (eligible to teach in a particular content area in grades six through twelve) 
and fewer had multiple subject credentials (eligible to teach all subjects in grades pre-K through 
eight) compared to the non-hired teachers. 

RQI1: Coding Applicant Essays to Identify Most Common Themes 

In addition to many standard application elements, the district’s application asks for 
responses to three short-answer essays detailing how applicants would address particular social 
issues and problems of practice that are relevant for teaching in the district context. Because of 
the district’s equity emphasis, we focus solely on the following short-essay question: 

Superintendent [Name] has stated that the achievement gap is the greatest civil 

rights issue facing our district today and closing that gap is the foundation and 

vision for the critical work of our teachers, staff, and administrators every day. As 

an educator, what is your role in working towards closing the achievement gap in 

[district]? In considering the demographics of our student population, what 

experiences or skills make you well-positioned to close the achievement gap in the 

context of a diverse district such as [district] ? 


This study uses responses to this question to indicate job candidates’ attitudes towards equity and 


approaches to addressing inequality. Notably, the district leaves the specific meaning of “the 


3 The applicant pool is less diverse than the current teaching force in the district where approximately 50 percent of 
teachers identify as non-white. 
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achievement gap” up to the interpretation of the applicant, not defining priorities about racial 
achievement gaps, income achievement gaps, or simply gaps between high- and low-achievers. 

We use Structural Topic Modeling (STM), an unsupervised machine learning technique, 
to automatically detect the most common topics applicants discuss and their distributions across 
all of the essays (see, e.g., Roberts, Stewart, and Tingley 2014). STM assumes each document is 
a mixture of topics, and each topic is a mixture of a set of representative words. STM is 
particularly useful for evaluating massive textual data, as in our case, and the output of STM can 
provide metrics about the text content to be used in quantitative analysis. 

Before we estimated the STM, we conducted standard pre-processing to prepare the texts 
for analysis. We first removed all stop words and punctuation.* Then we transformed all the 
words to lower case and reduced words to their root form, a procedure called stemming. As an 
unsupervised method, STM requires us to set the number of topics before we run the model. The 
optimal number of topics needs to balance model fit and substantive interpretation. We estimated 
the model multiple times to identify the ten, fifteen, twenty, and thirty most common, unique 
topics. We also controlled for whether an applicant previously worked in the district and their 
total years of K-12 teaching experience in our topic estimation models. 

Our research team then examined the output from each model to classify the topics into 
essay themes. Four human coders from our research team examined model output, including top 
words, the most common words for each theme; FREX, the frequency and the exclusivity of each 
word to each theme; lift, which weights words by their frequency in a specific topic and other 


topics; and score, the top words from an index commonly used in Linear Discriminant Analysis. 


4 Stop words are considered the most common words in a language, such as the, about, and own. In addition to 
removing these words, we remove many of the words used in the question itself, particularly the name of the 
superintendent. 
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Coders independently reviewed example essays that were the most emblematic of each topic to 
assign a topic label. From this we discussed disagreements as a team and developed harmonized 
theme labels for the various topics within each coding set. Many model themes were consistent 
regardless of the number of distinct topics. In models with fewer topics, several of the themes 
identified in the more numerous coding schemes combined into a single theme. The coders agreed 
that the most concise variant with only ten distinct topics captured the majority of the themes from 
across the variants with a larger number of topics. Diagnostic measures of essay topic classification 
(not shown) indicated that we lose little information by using ten topics rather than fifteen, twenty, 
or thirty. In addition, our interrater reliability of theme naming was highest with the smallest set 
of topics (80 percent exact agreement on labels with ten topics as opposed to 50 percent with 20 
topics). We use this model with ten themes to describe the prevalence of each theme across the 
pool of applicant essays and use the ten themes identified as key predictors in our analyses. 

In addition to identifying the most prevalent themes across the corpus of application 
essays, the STM algorithm codes each essay for the proportion of the text that addresses each 
theme and gives a rating of the density with which the essay covers each theme. We standardize 
these proportions with which applicants address each theme. We use these standardized theme 
values in regression models examining which applicants write most about which themes and 
which themes predict hiring outcomes most strongly. Theme ratings are not mutually exclusive 
and a given essay can have elements that correspond to multiple themes simultaneously. 

Because the district’s evaluation of an essay might be influenced by other features of the 
writing, we measure several characteristics of writing quality, including: (1) sentiment; (2) 
readability; (3) lexical density; (4) number of misspelled words; and (5) text length. We 


evaluated essays based on whether their overall sentiment or tone is more positive or negative 
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using three off-the-shelf dictionaries to score the sentiment. Correlations between the sentiment 
measures were fairly high (r = .73-.78), and thus we include only the Syushet sentiment scores 
(Jockers 2017). Readability, or understandability, refers to the level of education one needs to be 
able to read a piece of text easily. Readability scores are calculated using a grade level scale, 
which roughly corresponds to the number of years of education one needs to read a given text. 
We tested the readability of the essays using the four most popular readability measures. 
Correlations between the readability measures were very high (r = .86-.97), and thus we control 
for only the Automated Readability Index in our analysis (Senter and Smith 1967). Lexical 
density refers to the number of lexical, or content, words in a sentence divided by the total 
number of words. Lexical words are those that give a sentence its meaning including the nouns, 
verbs, adjectives, and adverbs. A high lexical density indicates a large number of information- 
carrying words, which is generally harder to read. Lexical density is negatively correlated with 
the length of a text. We use a type-token ratio (TTR) to measure the lexical density of the 
applicant essays (Templin 1957). Finally, because essay screeners may be influenced by overall 
essay length and obvious errors, we control for the total number of words applicants used in their 
essays and the percent of those words that are misspelled. 
RQ 2: Examining Variation in Essay Theme Content 

After identifying the most common themes discussed in applicants’ essays, we test 
whether theme coverage varies by race/ethnicity. Because we are interested in whether 
applicants of color and white applicants share similar beliefs about solutions for inequality and 


achievement gaps, we focus our discussion of the results on this dimension.” To examine how 


° We also test for differences based on variation among other demographic, experience, and quality dimensions (not 
shown). 
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essay themes vary by race, we estimate OLS regression models predicting the degree to which an 
essay includes content about each of the ten themes. These models take the form: 

Tint = a+ Race/Ethnicity Bi, + Xit +V¥e+ Fit C1) 
Where Tjn;¢ is an indicator for the proportion of applicant i’s essay that covered a specific theme 
n, in year t which is a function of their race/ethnicity £;,, a vector of controls X;, for other 
applicant demographics, qualifications, and experience, year fixed effects y,, and an error term. 
We estimate separate models predicting variation in the coverage of each of the ten themes. 
RQs 3 & 4: Application and Hiring Outcomes 

To be hired in the district, an applicant must first upload all of their application content, 
including their essays, and then apply to a particular position. Applicants may submit 
applications to multiple positions. Over 83 percent of applicants applied to two or more jobs, but 
they used the same application for all district positions to which they apply. 

The essay responses are likely indicative of some type of social desirability bias on the 
part of the applicant given that they are seeking employment. This is likely true of all data 
collected about applicants because of the very nature of seeking a job. This might lead them to 
write responses that they believe will elicit specific reactions on the part of principals in the 
district. However, because applicants cannot change their essay, they are unlikely to write 
content that is specifically tailored to different positions. Moreover, job postings for specific 
schools do not reference attitudes or values that principals are seeking; typically, an applicant 
applying to multiple positions would simply see variation in the job description or subject area 
needed for each posting to which they submit an application. 

We do not have a good understanding of what motivates educators to apply for some 


positions and not others. While prior research indicates that teachers move to higher-achieving, 
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whiter, more-advantaged schools over time, it is also possible that their beliefs and attitudes 
influence the initial school placements they seek in a district (Boyd et al. 2005). To test this, we 
examine the degree to which writing about particular essay themes predicts whether applicants 
apply to schools with student populations that are above the annual district average for different 
characteristics. We code each school as having student populations that are above district 
averages for three characteristics based on the district’s public reports of school composition: 
race/ethnicity, English Language Learners (ELLs), and free/reduced price lunch enrollment. To 
examine how applicant essays predict whether applicants apply to schools with different student 
populations, we estimate a series of linear probability models that take the basic form: 
Yi, = a+ Essay Elements Bit + Xit +Ve + €it (2) 

where Y;+ is an indicator for whether applicant i applied to a school with a student population that 
was above the district average in that year (e.g., above average percent ELL) in year t which is a 
function of several essay elements £;; including essay themes, indicators of writing quality, and 
essay scores, depending on the model, a vector of controls X;, for applicant demographics, 
qualifications, and experience, year fixed effects, y; , and an error term. In this analysis, we 
exclude applications that individuals submit to positions that are not located at a specific school 
site.® 

After applicants submit their application to a specific position in the district through the 
online system, staff in human resources conduct an initial screening of the application. This 
consists of reviewing the materials for completeness, confirming that the appropriate certification 
documents and transcripts have been submitted with the application, and an initial review and 
5 Supplemental models examine all applications per applicant and include fixed effects for the general job category 
(e.g., social studies teacher, counselor, bilingual Chinese teacher), or fixed effects for the specific position an 


applicant applied to. As the results are qualitatively similar, we present models in the paper without the subject area 
or job-specific fixed effects. Alternative versions are available upon request. 
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scoring of the application essays. This screening is done with a rubric (not shared to preserve 
district confidentiality) that evaluates each of the three essays for specific criteria and 
competencies, awarding zero to three points for no evidence, mixed/limited evidence, satisfactory 
evidence, or strong evidence of each competency. These scores are then aggregated across 
essays. Applicants with a score of four or lower are removed from the application pool and 
applicants with scores of five or higher are passed on to an internal database for principals and 
central office administrators to review. 

We use the rubric scores on each of the three application essays as an indicator of 
applicant quality. We create a global score for the three essays, we use the score on the 
achievement gap essay as a stand-alone, and we aggregate the two non-achievement gap essay 
scores for control variables in some models. Mean scores for these essays for the full sample and 
separately for hired and non-hired applicants are shown in Table 1. 

To examine whether discussing essay theme predicts rubric scores or being hired by the 
district, we estimate analogous models to equation (2), where rubric score or hired is the 
dependent variable. The same set of predictors is included in these models. We fit the same 
models as (2) to predict whether applicants are hired by schools with student populations that are 
above reported district averages across several demographic categories. 

Results 
Most Common Themes. Educators provide a great variety of responses to the essay question, 
suggesting that educators have meaningfully different values and attitudes about the social and 
achievement inequities facing district students regardless of the image they attempt to project 
towards their prospective employer. The content covered and omitted, as well as the general 


tenor, argumentation, and structure help to classify applicants based on their perspectives on the 
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underlying causes and challenges involved in addressing achievement gaps. Clear distinctions 
appear in the approaches to addressing this challenge and in the attitudes and biases held by these 
applicants. 

We summarize this rich detail into the ten most common themes that emerge from our 
structural topic model coding of applicants’ achievement gap essays. Table 2 lists these ten 
themes. Each label summarizes the approach that educators espouse to complete the phrase, “use 

to reduce inequality.” For example, some applicants would “use special services to 
reduce inequality and close achievement gaps.” Example texts most emblematic of each of the 
ten themes are in Appendix A.’ 

Several themes describe pedagogical approaches and experiences that applicants believe 
equip them to address achievement gaps. Two themes speak to the ways in which applicants 
adapt and customize their instruction to the needs of diverse learners by using differentiated 
instruction and cross-subject strategies, integrating content across subject areas and adapting it 
for different skill-levels to reduce inequality. One theme draws heavily upon recent school 
reform approaches and advocates using standards and assessment to improve and monitor 
instruction thereby reducing inequality. Another argues that educators are best equipped to 
reduce inequality by drawing upon their personal and instructional experiences with cultural and 
linguistic diversity. 

Other applicants focus more on creating a supportive, inspiring classroom and engaging 
school, family, and community resources in their efforts to reduce inequality. Two of the themes 
describe resources outside the classroom that the applicant would access. Applicants who 
indicate that they would access special services write about working with school services such as 


7 Applicants could cover multiple themes in their essays, although correlations between themes (not shown) were 
low to moderate, and the largest correlations of approximately .45 were negative. 
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therapy and speech pathologists to support diverse learners. Those who discuss family and 
community engagement write about working with the broader community and families and 
actively inviting them into the school. Essays also describe a more general idea of having a 
supportive classroom rather than referring to a specific strategy. Another theme focuses on being 
supportive, but relies heavily on general ideas about inspiring and believing in students to 
overcome challenges without clearly connecting these ideas to specific behaviors. 

Finally, two themes highlight the societal power structures that create and maintain 
unequal conditions and achievement gaps in the district. Essays naming structural causes detail 
the structural challenges facing marginalized students and the ways societal inequities contribute 
to achievement and opportunity gaps, arguing that these inequities must be interrogated and 
addressed to make progress in reducing inequality. In some cases, this theme also includes 
language typically associated with deficit orientations, emphasizing group and family conditions 
that reinforce societal inequities. A similar theme identifies societal causes for achievement gaps, 
but discusses educators’ responsibilities to work through these challenges to reduce inequality. 
Educators discussing this social-justice oriented theme decry opportunity gaps, want to challenge 
biases, and discuss drawing on students’ funds of knowledge and using critical and culturally 
relevant pedagogies to promote equity. 

Table 1 includes the average fraction of the essay text that discusses each topic and 
comparisons of the degree to which hired and non-hired applicants’ essays discuss each of the 
ten topics. Tests of differences in the use of each topic by group are also shown in Table 1, 
suggesting significant differences in topic use between hired and non-hired applicants on nearly 


every essay topic. Table 1 also includes descriptive comparisons of the other essay features 
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including readability, lexical complexity, and number of words for the full sample and between 
hired and non-hired applicants. 

On average, the degree to which essays address each theme may seem low, but this is in 
part because each essay also includes text that is either not specific enough to correspond to a 
particular theme or addresses another topic that was not part of these ten most common themes. 
The most prevalent themes are supportive classroom (covered in 17 percent of essays) and family 
and community engagement (covered in 14 percent of essays). The least prevalent theme is 
cross-subject strategies, which is covered in only 5 percent of essays. 

In addition to variation in percent of applicants who cover each theme across the body of 
essays, applicants also vary in the degree of their essay text that is devoted to covering a specific 
theme. This ranges from educators’ responsibilities, about which an essay has as much as forty 
percent of its content covering this theme to standards and assessment about which an essay has 
as much as sixty-eight percent of its content about this theme. As noted elsewhere, applicants 
often have some coverage of multiple themes in their essays, frequently with one theme 
dominating the text and one or more others occupying less of the overall essay text. 

Figure 2 presents a graphical representation of theme density within the entire corpus of 
essays, Showing how two themes overlap, and displays portions of example texts that are among 
the most emblematic of standards and assessment and educators’ responsibilities. The essay 
emblematic of the standards and assessment theme indicates the applicant’s strong belief in 
standards-aligned curriculum and lesson plans, frequent assessment of student progress, and 
accountability to help address the achievement gap. The example educators’ responsibilities 
essay highlights the need to use culturally relevant teaching and learning and to validate student 


cultures and experiences, and advocates for systemic, justice-oriented reform. These applicants 
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clearly have different values and beliefs about educators’ roles in addressing inequality and 
preferred strategies for addressing achievement gaps. 

[Figure 2 about here] 
Variation in Theme Content by Race. Essay themes vary by the race/ethnicity of the 
applicants. Table 2 displays results from models predicting theme content based on applicants’ 
race/ethnicity, controlling for other demographic characteristics, experience, and credentials. A 
number of differences emerge. Applicants from different race/ethnic groups cohere around 
particular sets of themes, relative to white applicants. Black applicants focus more on naming 
structural causes of inequality, such that their essays include .23 standard deviations more text 
about structural inequities than do white applicants’ essays. They also call for greater educator 
responsibility than white applicants do. Asian applicants tend to focus more than white 
applicants on themes that engage community, family, and special services and build supportive 
classrooms. Compared with white applicants, Hispanic applicants focus more on creating 
supportive classrooms, invoke their experiences with cultural and linguistic diversity, and frame 
their approaches to the achievement gap by naming the structural causes of inequality and 
identifying educators’ responsibilities to address them.® 

[Table 2 about here] 
Application and Hiring Outcomes. 

Applications: Applicant essays are predictive of candidates’ application decisions. Table 

3 presents results from models predicting the characteristics of schools to which applicants 
apply. These models include controls for the applicants’ demographic characteristics, experience, 
credentials, and measures of writing quality. In each case, the outcome is whether or not the 


8 There are also notable differences by gender, credential type, experience, and education. Results available upon 
request. 
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applicant applied to a position in a school that was above the district average demographic 

composition for that student group in that year. Given the specificity and ubiquity of 

differentiated instruction in teacher preparation programs, widespread use in classrooms, and 

because it is the theme that most closely resembles a solution for gaps between high- and low- 

achievers rather than race or income gaps, it serves as the reference category theme throughout. 
[Table 3 about here] 

Table 3 shows notable differences in the essay topics covered by applicants to schools 
with different demographic compositions. In particular, applicants who apply to schools that 
have above-average Hispanic populations write more about nearly every theme compared with 
differentiated instruction, except for family & community engagement. A similar pattern is 
evident among applicants to above-average Asian schools, although the coefficients are only 
significantly different for standards and assessment. Applicants to above-average ELL schools 
focus most on experiences with cultural & linguistic diversity and standards and assessment, 
which is consistent both with the needs these schools have and the recent policy prescriptions for 
schools whose students struggle with English proficiency. Applicants who apply to above- 
average free/reduced-price lunch schools discuss themes somewhere between those who apply to 
above-average Hispanic schools and above-average ELL schools. Only one theme, naming 
structural causes, significantly predicts applying to above-average black schools. 

Scores: After applicants submit their application, screeners in the district’s human 
resources department read it and score each of the three essays using an internal rubric. In 
models (not shown) we examine the relationship between overall scores and specific essays 
scores. Higher essay scores positively predict being hired, but the achievement gap essay score 


matters much more for being hired than scores on the other two essays. An additional point on 
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the achievement gap essay increases an applicant’s likelihood of being hired by 6.6 percentage 
points whereas the other two essays combined only increase the likelihood of being hired by 3.7 
percentage points (22 percent of applicants were eventually hired). This suggests that there is 
additional information in the achievement gap essay that benefits candidates beyond simple 
evaluation scores. Given the importance of the essay scores and themes, Table 4 displays results 
from models examining the relationship between essay themes, essay scores given by the initial 
application screening, and being hired by the district. 

[Table 4 about here] 

Discussing almost every one of the themes is associated with higher essay scores than 
discussing differentiated instruction (except family & community engagement and believing in 
students to overcome). This suggests that the district prioritizes many other beliefs and responses 
to achievement gaps over something like differentiated instruction that is part of typical educator 
training and practice. The themes that are scored most highly are educators’ responsibilities and 
experience with cultural/linguistic diversity, but standards & assessment and supportive 
classroom are also scored fairly highly. This pattern holds even with controls for race/ethnic 
identity and is thus not an artifact of the district simply preferring candidates of color who are 
more likely to discuss these themes in their essays. 

Hiring: Many of these themes are also associated with being hired in the district. In 
general, the themes that received higher scores in the initial screening process are associated with 
the greatest likelihood of being hired. While the coefficients might seem small, it is important to 
consider them relative to the means and standard deviations for the sample. For example, the 
average achievement gap essay score was 1.9 (on a 0-3 scale) with a standard deviation of .8. By 


discussing educators’ responsibilities more than differentiated instruction an applicant increases 
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their achievement gap essay score by .1 points, which is thirteen percent of a standard deviation. 
Moreover, that same content increases the likelihood that they are hired by 2.5 percentage points, 
regardless of specific position sought, which is roughly a ten percent increase in the likelihood of 
being hired. Additional models (not shown) omit different themes to serve as the reference 
category to test the relative importance of each theme. While writing about almost any of the 
themes improves the likelihood of getting hired more than discussing differentiated instruction, 
two themes relate to the highest likelihood of being hired relative to all other themes, experience 
with cultural/linguistic diversity and educators’ responsibilities. The two themes are not 
significantly different from one another, although the coefficients for educators’ responsibilities 
are positive relative to those of cultural/linguistic diversity. 

An additional concern beyond whether educators’ attitudes and values predict being hired 
by the district is the distribution of educators across schools. In particular, if districts think that 
having educators with particular pedagogical orientations and experiences might be better at 
promoting equity, they might also want to concentrate such individuals in traditionally 
underserved schools. However, while district administrators might prefer this, such candidates 
might be attractive to all types of schools and by virtue of a decentralized hiring process, may 
end up with placements at schools with relatively more advantaged students. Table 5 presents 
results from models investigating whether applicant essay themes are differentially associated 
with being hired in schools with different student populations, conditional on having applied. As 
in Table 3, the outcomes in this table are whether the school in which the educator was hired has 
above district-average concentrations of students from particular demographic groups. 


[Table 5 about here] 
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The results in Table 5 have some patterns in common with Table 3, but with a few 
notable differences and smaller coefficients. Two themes positively predict being hired in 
schools with large concentrations of many different types of students, special services and 
educators’ responsibilities. Only one other theme, experience with cultural/linguistic diversity, 
predicts a higher probability of being hired in schools with above average Asian populations, 
while supportive classroom predicts higher probability of being hired in schools with above 
average numbers of FRPL-enrolled students. As with the likelihood of applying, writing about 
nearly every essay theme increases an applicant’s likelihood of being hired in schools with 
above-average populations of Hispanic students compared with writing about differentiated 
instruction, except family & community engagement. Although writing about structural causes is 
the only theme to significantly predict applying to above-average black schools, successful 
applicants to the same schools discuss a blend of classroom climate themes—family & 
community engagement, supportive classroom, and special services—and educator strategy 
themes standards & assessment and educators’ responsibilities.° 

Discussion 

This study uses school district administrative data to examine how educators’ beliefs and 
attitudes about inequality discussed in application essays impact application behavior and hiring 
outcomes. It uses machine learning techniques to identify the most common themes across over 
ten thousand essays and tests whether applicants that write about particular themes are more 


likely to apply to schools with certain student populations, are rated more highly by the district, 


° One might worry that prior experiences in the district or in the specific school to which a teacher was re-applying 
might lead them to write especially well-tailored essays. Supplemental results indicate that the themes discussed do 
vary somewhat between prior employees and completely new applicants. However, similar themes predict higher 
screening scores and successful hiring outcomes regardless of prior employment in the district or specific school. 
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and are more likely to get hired. This multifaceted investigation makes three contributions. First, 
it extends work examining current and pre-service educators’ attitudes about inequality by 
investigating equity attitudes among a very large sample of educator applicants. Second, it 
contributes to literature evaluating educator-hiring processes and hiring in other settings to 
consider the ways in which attitudes expressed by applicants in their applications impact hiring 
outcomes. Third, it harnesses existing administrative data and uses text-as-data empirical 
techniques to glean new knowledge about beliefs and attitudes, which are difficult to measure 
and evaluate systematically. 

Although one might worry that the essays do not reveal applicants’ true beliefs, attitudes, 
and values about equity and achievement gaps, the application essays do provide insight into 
applicants’ perceptions of what they think an employer would like to hear about such topics. 
With the exception of a few applicants that provide very brief responses, most applicants write 
thoughtful statements about how they might address a very real problem facing the district. The 
wide variety of responses suggests that educators have differing perspectives about how to 
address inequality, even when trying to impress an employer. 

The essays show pronounced differences in themes which are differentially related to 
hiring outcomes. Distinct profiles emerge that vary by race of the applicant as well as the student 
population they aim to serve. Black and Hispanic applicants—groups that have been traditionally 
marginalized in educational settings—write more about issues around structural inequities facing 
students, advocate for solutions that stress educators’ responsibilities for addressing inequities, 
and draw on experiences with cultural and linguistic diversity. Many of these same themes are 
positively related to essay scores and hiring outcomes, regardless of the race of the applicant, 


compared with differentiated instruction, perhaps because differentiated instruction among 
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marginalized populations may be perceived as a way to separate, track, and withhold educational 
opportunities. Some of the themes also predict an applicant’s interest in teaching in schools with 
marginalized student populations, but a somewhat different set are associated with actually being 
hired in such schools. In contrast, Asian and white applicants—groups that have traditionally 
been advantaged in this district and schools more broadly—focus more on family & community 
engagement and differentiating instruction. Applicants seeking to work in schools with more- 
advantaged students talk less about structural inequity and instead focus on meeting individual 
student needs through differentiated instruction. However, almost all of the other themes are 
more strongly related to being hired than differentiated instruction. In other words, educators’ 
approaches to this issue, and their underlying values, attitudes, and beliefs that inform their 
responses matter in district hiring decisions. 

The degree to which educators cultivate distinct equity-oriented personas or profiles in 
response to this question is striking. It parallels work by sociologist Lauren Rivera (2016) that 
examines how students in elite colleges work to cultivate the cultural capital needed to 
successfully match to high-status private sector jobs. In our setting, educator applicants work to 
brandish not their elite cultural bona fides, but to demonstrate that they possess dispositions that 
position them to work as change-agents in a district seeking to combat systemic inequality 
(Bourdieu 2000). Yet, although there are multiple profiles that appear to be desired by the 
district, not all applicants are equally successful at projecting this particular image. Educators 
from underrepresented racial backgrounds are particularly good at characterizing themselves in a 
manner that is consistent with the district’s social justice aims, but many white applicants also 


reflect thoughtfully on the ways they would address inequality, and the district appears to value 
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this change-agent cultural capital regardless of the demographic background of the applicant who 
displays it. 

The limitations of this study should be kept in mind when interpreting our results. In 
particular, we do not observe all stages in the hiring process. Between the district’s initial 
screening and the time that applicants are hired for a position, hiring procedures are largely left 
to the discretion of individual principals. This makes it difficult to determine what type of 
interview process occurred and how much additional information principals had beyond the 
application materials with which to make their hiring decisions. This opacity makes it difficult to 
determine what specific information principals used to select one candidate over another. 
Another limitation is that while structural topic modeling does allow for a careful screening of a 
large volume of text, the algorithms may overlook important insights that human readers would 
identify. Human coders might make meaning out of particular phrases or topics that machine- 
learning techniques cannot. 

Finally, this study takes place in a district that is extremely cognizant of structural 
inequality and particularly mindful of its role in creating and combating these inequities. Not all 
districts have this focus or awareness. Thus, the types of candidates attracted to work in this 
setting and the types of responses applicants provide to this prompt are likely different than they 
would be if many other districts asked for the same information. While the generalizability of 
these results might be limited, they do provide helpful insights about what individuals think (and 
the variability in what individuals think) about the achievement gap and equity in a setting that 
brands itself as being actively engaged in the work of trying to combat it. Many U.S. school 
districts aim to develop a workforce that puts equity at the center of their work; these results 


indicate that applicants do have different ideas about how best to do so. Incorporating some type 
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of application essay on this topic can help discern important differences and can improve 
selection processes. Moreover, essay prompts discussing other educational challenges could be 
implemented to similar effect. 

This study identified important variation in applicants’ beliefs about how to address a 
major educational challenge: the achievement gap. It also revealed that such responses influence 
both application and hiring behaviors in this district. As in this study, data-mining techniques can 
unlock novel insights from text-rich administrative data and can inform conversations about 
educator hiring, diversity, retention, and many other policy-relevant issues. This method 
summarized complexities about educators’ beliefs, values, and preferences that are otherwise 
costly and challenging to collect across a large number of individuals. Such data could be further 
linked to student and educator records to examine relationships between applicant attitudes, 
student outcomes, and educator job performance and tenure—relationships we will address in 
our setting in future work. Similar data in other settings could be matched to other employee 
outcomes including performance metrics, satisfaction, and retention. By drawing insights from 
existing administrative data, this relatively new combination of data and methods has the 
potential to make novel contributions to general knowledge about human thought and behavior 


as well as innovative improvements in policy and practice, in education and elsewhere. 
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Figure 1: Number of job Postings, Applications, and Applicants Hired by Year, 2009-2015 
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Figure 2: Comparing Concentration of the Standards & Assessment and Educators’ 
Responsibilities Themes across All Essays. 
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Standards and Assessment 


Strong Standards & Asssessment Essay 


“First, educational standards at both the national and state level are necessary to closing the 
Achievement Gap. Administrators, teachers, and parents should be aware of the standards a 
student must meet to pass their grade level.” 


Strong Educators’ Responsibilities Essay 


“T not only have an obligation to provide my students with an equitable education, but also to be 
an advocate for change. We need to move toward a curriculum and pedagogy that is culturally 
responsive and relevant to our students...Closing the achievement gap, creating schools that are 
equitable and just, changing the way we view education, and ensuring that every educator 
reflects on their own experiences and biases, will take time and is a journey that I am completely 
dedicated and committed to. I strive to teach for social justice.” 


Table 1. Descriptive Statistics of Applicant Sample by Hired Status 


Variable Full Sample Hired Not Hired Group diffs 
mean sd mean sd mean sd p-value 
Hired 0.22 0.42 1 0 0 0 
Applicant Characteristics 
White 0.52 0.50 0.49 0.50 0.53 0.50 0.000 
Black 0.05 0.22 0.04 0.20 0.05 0.23 0.007 
Asian 0.14 0.35 0.17 0.37 0.14 0.34 0.000 
Hispanic 0.14 0.34 0.16 0.37 0.13 0.34 0.000 
Other race 0.04 0.20 0.05 0.22 0.04 0.20 0.048 
Decline to state race 0.10 0.30 0.10 0.30 0.11 0.31 0.177 
Female 0.67 0.47 0.72 0.45 0.65 0.48 0.000 
Years of prior k-12 experience 4.80 6.29 4.58 5.78 4.87 6.43 0.032 
Number of jobs on application 4.22 2.59 4.24 2.51 4.21 2.62 0.637 
Multiple Subject Credential 0.36 0.48 0.41 0.49 0.34 0.47 0.000 
Special Ed Credential 0.19 0.40 0.22 0.41 0.19 0.39 0.000 
STEM credential 0.15 0.36 0.15 0.36 0.16 0.36 0.341 
Humanities credential 0.10 0.30 0.07 0.26 0.11 0.32 0.000 
Foreign language credential 0.02 0.15 0.02 0.14 0.02 0.15 0.167 
Social science credential 0.07 0.26 0.06 0.23 0.08 0.27 0.001 
Physical Ed 0.02 0.13 0.02 0.15 0.02 0.12 0.011 
Early Childhood Credential 0.02 0.13 0.01 0.11 0.02 0.13 0.034 
Administrative Credential 0.01 0.08 0.01 0.07 0.01 0.09 0.249 
Supplemental Credential 0.03 0.17 0.03 0.18 0.03 0.17 0.801 
Substitute Credential 0.03 0.18 0.01 0.12 0.04 0.19 0.000 
GPA 3.02 0.99 3.16 0.85 2.98 1.03 0.000 
No degree declared 0.03 0.17 0.02 0.13 0.04 0.18 0.000 
Associates Degree 0.00 0.05 0.00 0.05 0.00 0.06 0.344 
Bachelor's Degree 0.40 0.49 0.39 0.49 0.41 0.49 0.056 
Master's Degree 0.53 0.50 0.57 0.50 0.52 0.50 0.000 
Ph.D. 0.03 0.18 0.03 0.16 0.03 0.18 0.033 
N applications submitted 22.97 40.94 23.33 40.73 22.87 40.99 0.591 
Essay Themes 
Special services 0.08 0.08 0.08 0.09 0.08 0.08 0.000 
Family & community engagement 0.14 0.09 0.12 0.08 0.14 0.10 0.000 
Believe to overcome (platitudes) 0.10 0.08 0.09 0.08 0.10 0.08 0.000 
Experience with cult/ling diversity 0.09 0.09 0.10 0.09 0.08 0.08 0.000 
Standards & assessment 0.11 0.10 0.11 0.09 0.11 0.10 0.274 
Educators’ responsibilities 0.08 0.06 0.09 0.07 0.08 0.06 0.000 
Supportive classroom 0.17 0.10 0.17 0.09 0.17 0.10 0.006 
Naming structural causes 0.08 0.08 0.08 0.07 0.08 0.08 0.006 
Cross-subj. strategies 0.05 0.07 0.05 0.07 0.05 0.07 0.342 
Differentiated instruction 0.11 0.09 0.11 0.08 0.11 0.09 0.023 
Writing Quality 
Sentiment measure: Syuzhet 12.02 6.36 13.26 6.21 11.67 6.36 0.000 
Automated Readability Index 12.89 3.50 13.20 3.18 12.81 3.58 0.000 
Type-Token Ratio (lexical complexity) 0.59 0.10 0.57 0.08 0.59 0.10 0.000 
Number of words 241.33 126.71 26483 129.24 23464 125.18 0.000 
Number of misspelled words 1.14 2.47 1.31 3.18 1.09 2.22 0.000 
Total score: Application Essays 5.97 73 6.63 1.55 5.76 1.73 0.000 
Essay score: Achievement gap essay question 1.89 0.81 2.19 0.73 1.79 0.81 0.000 
N (Unique individuals) 13016 2883 10133 


The total number of unique submissions is 164,367. Authors' tabulations. 


Table 2. Predictors of themes 


Experience 
Family & with cultural Naming __Use cross- 
Special community Believeto & linguistic Standards & Educators’ Supportive structural subject Differentiated 
services engagement overcome diversity assessment responsibilities classroom causes Strategies _ instruction 
Black 0.018 -0.017 0.012 -0.030 0.063 0.089* -0.095* 0.231*** -0.129** -0.112** 
(0.037) (0.039) (0.040) (0.035) (0.040) (0.040) (0.040) (0.040) (0.040) (0.040) 
Asian 0.085* ** 0.178*** = -0.110*** 0.025 -0.078** -0.031 0.090* ** -0.022 -0.152*** -0.046 
(0.024) (0.025) (0.026) (0.023) (0.026) (0.026) (0.026) (0.026) (0.026) (0.026) 
Hispanic 0.011 -0.021 -0.055* 0.097*** -0.033 0.075** 0.071** 0.101*** -0.138*** -0.111*** 
(0.024) (0.026) (0.026) (0.023) (0.026) (0.026) (0.026) (0.026) (0.026) (0.026) 
Other race 0.034 -0.071 -0.003 0.073 -0.031 0.119** -0.036 0.130** -0.040 -0.121** 
(0.040) (0.042) (0.043) (0.038) (0.043) (0.043) (0.043) (0.043) (0.043) (0.043) 
Decline to state -0.003 -0.038 -0.126*** -0.021 -0.010 0.085* 0.042 0.102** 0.048 -0.045 
race (0.035) (0.037) (0.038) (0.033) (0.037) (0.037) (0.037) (0.038) (0.038) (0.038) 
Constant -0.180* 0.848*** 0.549*** = -0,459*** -0.376*** -0.474*** 0.341***  -0.401*** = -0,243** 0.120 
(0.072) (0.075) (0.078) (0.068) (0.077) (0.077) (0.077) (0.078) (0.078) (0.078) 
Observations 13016 13016 13016 13016 13016 13016 13016 13016 13016 13016 
12 0.194 0.106 0.050 0.270 0.064 0.065 0.065 0.042 0.052 0.050 


Models number of applications submitted. Themes standardized to have a mean of 0 and SD of 1. Authors' tabulations. 
All models include controls for teacher demographics, education, experience, credentials, and year in which the applicant applied. 
* p < 0.05, ** p < 0.01, *** p < 0.001 


Table 3. Do essay themes predict applying to schools with particular characteristics? 


Special services 

Family & community 
engagement 

Believe to overcome 
Experience with cultural & 
linguistic diversity 
Standards & assessment 
Educators' responsibilities 
Supportive classroom 
Naming structural causes 
Use cross-subject strategies 


Constant 


Observations 
r2 


Race is white and Multiple subject credential are omitted categories. Standard errors clustered by applicant. 


(1) (2) (3) (4) (5) (6) 
Location Applied to is above the District Average Percent Student Enrollment 
Black Hispanic Asian White ELL FRPL 
-0.002 0.007 0.007 -0.011** -0.001 0.001 
(0.003) (0.004) (0.004) (0.004) (0.004) (0.005) 
-0.000 0.004 -0.004 0.002 0.007 0.001 
(0.003) (0.003) (0.004) (0.003) (0.004) (0.004) 
0.006 0.010** -0.001 -0.010** -0.003 0.009* 
(0.003) (0.004) (0.004) (0.003) (0.004) (0.004) 
-0.003 0.016*** -0.006 -0.003 0.016*** 0.008* 
(0.003) (0.004) (0.004) (0.003) (0.004) (0.004) 
0.005 0.014*** -0.009* -0.006 0.009* 0.010* 
(0.003) (0.004) (0.004) (0.003) (0.004) (0.004) 
0.002 0.009** -0.003 -0.001 0.003 0.001 
(0.003) (0.004) (0.003) (0.003) (0.003) (0.004) 
0.000 0.012*** -0.004 -0.005 0.005 0.007 
(0.003) (0.004) (0.004) (0.003) (0.004) (0.004) 
0.005* 0.009** -0.006 -0.008** 0.001 0.011*** 
(0.002) (0.003) (0.003) (0.002) (0.003) (0.003) 
-0.002 0.011*** 0.001 -0.003 0.003 0.007 
(0.003) (0.003) (0.003) (0.003) (0.004) (0.004) 
0.498*** 0,394*** 0.489*** 0.321" 0:637""* 0.608*** 
(0.030) (0.037) (0.036) (0.030) (0.036) (0.038) 
130551 130551 130551 130551 130424 99651 
0.027 0.025 0.060 0.039 0.047 0.055 


All models include controls for writing quality, teacher demographics, education, experience, credentials, and year in 

which the applicant applied. Schools missing on school characteristics omitted from the analysis. Models also exclude 
applications to dentralized district positions. Authors' tabulations. 
* p< 0.05, ** p < 0.01, *** p < 0.001 


Table 4. How essay themes predict achievement gaps essay scores and being hired 


() (2) 
Essay score Hired 
Special services 0.058*** 0.011* 
(0.012) (0.005) 
Family & community engagement 0.014 -0.004 
(0.012) (0.005) 
Believe to overcome -0.009 0.002 
(0.012) (0.005) 
Experience with cultural & linguistic diversity 0.099*** 0.026*** 
(0.013) (0.006) 
Standards & assessment 0.079*** 0.018** 
(0.014) (0.006) 
Educators’ responsibilities 0.103*** 0.025*** 
(0.010) (0.005) 
Supportive classroom 0.078*** 0.016** 
(0.012) (0.006) 
Naming structural causes 0.057*** 0.009* 
(0.010) (0.004) 
Use cross-subject strategies 0.042*** 0.006 
(0.012) (0.005) 
Constant 1.570*** 0.095 
(0.133) (0.053) 
Observations 13016 13016 
r2 0.247 0.105 


Race is white and Multiple subject credential are omitted categories. 
Standard errors clustered by applicant. Authors' tabulations. 


All models include controls for writing quality, teacher demographics, education, 
experience, credentials, and year in which the applicant applied. Models also inlcude 
position-specific fixed effects. 

*p< 0:05," p< 0.01, *** p< 0,001 


Table 5. Do essay themes predict teacher hire in schools with particular characteristics? 


(1) (2) (3) (4) (5) (6) 
Location Hired is above District Average Percent Student Enrollment 
Black Hispanic Asian White ELL FRPL 
Special services 0.003* 0.003** 0.003* 0.003** 0.004*** 0.003** 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Family & community -0.000 -0.000 -0.000 -0.001 0.001 -0.000 
engagement (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Believe to overcome 0.002* 0.002 0.001 0.002 0.001 0.001 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Experience with cultural & 0.001 0.002* 0.002* 0.001 0.003** 0.002 
linguistic diversity (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Standards & assessment 0.002* 0.003** 0.001 0.001 0.003*** 0.001 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Educators' responsibilities 0.003*** 0.004*** 0.002 0.003** 0.003*** 0.003** 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Supportive classroom 0.002* 0.003** 0.001 0.000 0.003*** 0.002* 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Naming structural causes 0.000 0.001 0.000 0.000 0.001 0.000 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Use cross-subject strategies 0.001 0.001 -0.000 0.000 0.002* 0.000 
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Constant 0.011 0.021* 0.011 0.011 0.017 0.014 
(0.009) (0.010) (0.010) (0.010) (0.010) (0.010) 
Observations 71165 66117 50676 41950 72651 61158 
r2 0.009 0.010 0.011 0.007 0.008 0.011 


Models condition on having applied to positions at one or more schools with the identified student population. 

Race is white and Multiple subject credential are omitted categories. Standard errors clustered by applicant. 

All models include controls for writing quality, teacher demographics, education, experience, credentials, and year in 
which the applicant applied. Schools missing on school characteristics omitted from the analysis. Models also exclude 
applications to dentralized district positions. 

Authors' tabulations. * p < 0.05, ** p < 0.01, *** p < 0.001 


Appendix A — Example Text for each Essay Theme 
Special services 


In order to engage all students in their learning process, it is my role as an educator to 1) 
establish a consistent, comfortable, all-inclusive, stimulating and trust-enhancing environment 
inside the classroom; 2) to maintain open and trusting ongoing communication with students, 
their families and caretakers in the classroom and an open-door policy; 3) maintain a schoolwide 
support system with other staff members, all students, and their communities; 4) modify 
curricula based upon the IEPs of students with special needs and ongoing data based on students' 
experiences, cultural and language background, interests, progress and behavior; 5) develop 
appropriate, ongoing, and viable rewards systems for individual students as well as student 
teams; 6) use caution in applying research-based interventions to extinguish undesirable 
behavior; 6) work closely with teaching team for optimal teaching and schoolwide environment. 


Family & community engagement 


I believe that it is essential to get the whole community involved in the education of our children. 
This is one of the main reasons I want to work in [district], because I live here. It is important to 
reach out to parents, do home visits, get them involved in school activities. If the entire family 
feels a part of the school setting then it will be easier for all of us to work together to close the 
achievement gap. 


Believe to overcome 


... Students will not here anything you are saying unless they believe you actually care about 
them. This comes from having a heart and spirit of humbleness and sincerity. You cannot fool 
these kids into believing what you are saying is truth unless they know you care about their well- 
being. Once you as a teacher have established in your heart that you are in this profession for the 
kids and not for any other reason then as a teacher your presence will convey and show this to 
the kids. Next you can present the content matter that you have a passion for. I believe it is the 
teachers job and goal to inspire these kids. There have been many methods and programs in our 
schools to try and do this, but one cannot inspired unless one is inspired. This means as a 
education you must be full of courage and strength exhibiting genuine love to these kids no 
matter how hard they may be on you as a teacher or how negative their personal situation is at 
home. As a teacher you must believe in yourself and the kids. Energy and love must flow from 
your being to help these kids because a lot of them do not have many people who really care 
about them.... 


Experience with cultural/linguistic diversity 


Being multi-lingual is essential to closing the achievement gap. Many [district] students speak 
one language at home and another language at school: English. I have spoken Spanish with 
parents, translators, students, and with my paraprofessionals. I have even learned some Chinese 
and Arabic to speak with my students with Speech needs. I think the District should provide 
condensed History lessons for all teachers of all the various cultures of our students: African- 


American, Cambodian, Vietnamese, Chinese, Latino, European and Southeast Asian cultures. 
Since I have travelled throughout Mexico, I have some understanding of the culture. Having 
studied American History, I have some idea of the African-American experience. 


Standards & assessment 


It takes an outstanding educator to meet the needs of students and work towards closing the 
achievement gap. An outstanding educator is constantly and consistently assessing student 
learning as well as involving students in assessing their own progress. Progressive educators use 
the information from ongoing formal and informal assessments to guide the content and 
methodology of instruction to meet the needs of individual students. Documenting and 
communicating progress to students, parents, and those who work with the student not only 
conveys to the student the importance of being a self-motivated learner, but also improves 
professional practice. 


Educators’ responsibilities 


...Additionally, we need to critically examine the practices that are in place within the 
educational system, despite our best intentions, the very practices that we use daily work against 
students from minority ethnicities and low SES backgrounds. Utilizing culturally responsive 
teaching is at the epicenter of closing the achievement gap. Acknowledging our students’ diverse 
backgrounds gives legitimacy to each student’s culture, which in turn creates a clear connection 
between home and school, which is crucial in promoting student achievement. When this 
approach to instruction is used, learning becomes more appropriate and effective, because 
instruction stems from, and responds to the students’ strengths and existing knowledge. Due to 
the fact that non-dominant discourses are often ignored or shamed inside classrooms, it is crucial 
that the voices of students with non-dominant discourses are given focus and emphasis. The 
very nature and spotlight of spoken academic language in the classroom, needs to be shifted to fit 
the voices of the students... 


Supportive classroom 


... It is truly critical for the students themselves to value education and be self motivated. 
However, the gap will not be closed by the students or the teachers alone; the gap will ultimately 
be closed by the joining of the school, the families, and the [district] community. By making 
education a community effort, students will feel supported and understand the importance of 
their education. Thus my main role, as an educator, is to facilitate the learning of the students. I 
truly believe in a positive classroom environment in which the students are free to express 
themselves and learn with a lowered affective filter. I will do this by promoting an encouraging 
learning environment and showing students the true life value of education... 


Name structural causes 
In his book Savage Inequalities the education writer Jonathan Kozol describes how urban public 


school systems in the 1990s struggled to serve the needs of the poorest families in the country, 
especially in predominantly African-American and immigrant communities. He cites research 


indicating that the factors most closely correlated with high achievement in school are family 
income and education level of parents. Thus inner city schools working to improve achievement 
face the greatest challenge in the nation. Kozol’s own research revealed the stark reality that 
those same schools are funded by the lowest expenditures per student and equipped with the 
poorest physical resources. He reports on both the deplorable conditions in many impoverished 
school districts and the contrasting profusion of resources available in neighboring wealthy 
districts... 


Use cross-subject strategies 


I believe that it is very important for the educator to come up with creative strategies that are 
specifically tailored to each student's needs...I designed and taught nine week project classes 
with integrated curriculum. The 6th, 7th, and 8th grade students signed up for the project class 
that they wanted to be in. (Student choice is very important in motivating the student to achieve.) 
One example of a project was Mystery Play. For Language Arts, the students read and wrote 
mystery stories. In Math, they did logic and secret codes. In Social Studies, they studied actual 
cases. In science, they studied forensics. Each project had a final challenge and in this project 
the final challenge involved writing, producing, and performing a who-dunnit mystery play using 
the elements that they had learned... 


Differentiated instruction 


I believe that the number one key component in closing the achievement gap in schools today is 
adapting instruction to tailor to the specific needs of individual students.... I try to create 
interesting lessons and differentiate instruction so I can reach each of my students...It is a well 
known fact that not all students learn in the same way. Other than basic learning style 
differences, some students have learning disabilities or impairments which can significantly alter 
the way they receive and process information. I make a conscious effort to be very aware of 
things that may cause students to learn differently, and I use that information to help differentiate 
my instruction and future lessons. One way that I differentiate my instruction is through the use 
of centers. I like to set up various learning centers during a lesson to provide a variety of learning 
experiences that will allow students to learn according to their individual learning styles. The 
centers can include a variety of direct instruction of material, hands on learning activities, 
historical documents that students can analyze, and a center where students take the knowledge 
they have gained and create something.... 


